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Topics 


•  Navy  AO  As 

•  Interoperability  Impact  on  System  & 
Mission  Capabilities 

•  Current  Challenges 


Purpose:  Describe  Interoperability  Implications  to  AOAs 


What  is  an  AOA? 


•  Analytical  basis  for  mission/program 

-  Acquisition  Decisions 

-  Key  Performance  Parameters 

-  Estimated  Costs 

•  Required  by  DoD  Instruction  5000.2 

-  Support  of  milestone  decisions 

-  Primary  Input  to  mandated  Program  Document 
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Requirements  And  Acquisition  Process 


SI 
A 

CINCs 

Threat  change 
Technology 


REQUIREMENTS 


Mission 

Need 

Statement 

(MNS) 


Materiel  Alternatives 


ACQUISITION  (ASN  RD&A^ 


^  Program  Decision  Meeting  ^ 
(PDM) 


Acquisition 

Decision 

.  Memorandum  (ADM)  . 


Measures  of 
Effectiveness 
and  Performance 
(MOE/MOP) 
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T est  and  Evaluation 
Master  Plan  (TEMP) 


%  Operational 
I  Perfomiance 
I  Parameters 
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Analysis  Of  Alternatives 


J 


Operational 
Requirements 
Document  (ORD) 


M _ ^ 

Perfoimarf^ 

Parameters 


Subiequent  Phases/ 
Mil^tone  Decisions 

Milestone  I  ^ 
Approved 
Acquisition 
Program 

Baseline  (APB)  y 
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•  Independent  assessment 

•  Methodology 

-  Adapted  to  address  key  acquisition  and  mission  technical  issues 

•  Several  materiel  alternatives 

-  Represents  Trade  Space 

-  One  alternative  represents  the  Status  Quo  extrapolated  to  the  future 

•  Evaluation  Factors 

-  Threat,  Costs,  Mission  Capabilities,  Schedule,  Risk 

-  Measures  of  Effectiveness  (MOEs)  and  Measures  of  Performance  (MOPs) 
Development  tailored  to  distinguish  among  alternatives 

•  Modeling 

-  Assumptions  based 

-  MOEs/MOPs  determine  level  of  modeling  required 

•  Results  must  relate  to  decisions  to  be  made 

-  Basis  for  discarded  alternatives  as  well  as  selected 

-  Value  to  mission  quantised 

-  Key  performance  and  cost  drivers  (goals  and  threshholds  inputs) 
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TBMD  AOA  Levels  of  Analysis 


High-Fidelity 

Element 

Characterization 


One-on-One 

Characterization 


Force-on-Force 


Number  of 
Interceptors 


Number  of 

Number  of 

Defense  Ships 

Leakers 

TBMs  Engaged 

Required 

Sensitivity  to 
Ship  Location 


Threat 

Coverage 


1 

Raid-Handl 

ing  Capability 

Blue  Attrition 

Defended  Asset 

Attrition 

Campaign 
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Interoperability  Relationships 


CJCSI  3170.01A,  “Requirements  Generation  System,”  of 
10  August  1999  requires  that  each  CRD  /  ORD  include 
Interoperability  as  a  KPP. 


•  If  AOAs  provide  the  analytical  basis  for  the  KPPs, 
then  Interoperability  must  be  addressed  by  the  AOA 


•  To  be  properly  evaluated,  interoperability  payoffs 
for  mission  capabilities  must  be  determined 
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Historically,  AOAs  and  other  Force  Analyses  have  NOT 
Evaluated  Effects  of  Interoperability 


7 


Topics 


Navy  AO  As 

Interoperability  Impact  on  System  & 
Mission  Capabilities 

Current  Challenges 


Purpose:  Describe  Interoperability  Implications  to  AOAs 
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OD 


Battle  System  Confusion 
Fleet  Air  Defense  Example 


•  Dual  Tracks 

•  Hostile  ID  errors 

•  Friend/Neutral  ID  errors 


Decision  Delays 
Decision  Errors 


Add  Challenging  Threat  And 
No  Time 


Increased  Vulnerability 
Reduced  Effectiveness 


Mission  Performance  Factors 


(Force  defense  Mission  Focus) 


Coverage  *  Availability  *  Detect/Track  *Eval/Decide  *Negation 


•  Environmental 

•  Threat 

•  Volume 


•  Operational 

•  Stationkeeping 

•  Capacity 


•  Detection 

•  Tracking 

•  Identification 


_ 1  1 _ 

•  Decision  Makini 

y 

9 

_ 1  1 _ 

•  Reaction  Time 

•  Firepower 

•  Coordination 

•  Lethality 

•  Kill  Evaluation 

-  Area/Situation  -  Force  Posture  -  Common  -  Decision  Making  -  Engagement 

of  Interest  -  Assets  to  Situational  -  Reaction  (&  Coordination 
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Effectiveness  (Attrition  &  Fratricide)  and  Efficiency 
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Battle  Force  Interoperability 
Measures  Hierarchy 

TAMP  Example 


MOEs 


MOPs 


Units  Needed  to  Win 

Time  Needed  to  Win 

Red  Losses  Blue  Units  Lost  Eratricide 

PRA  Kills 

Leakers 

Weapon  Efficiency 

Eratricide  losses 

Layers  Employed 

Engage  Pk 

Expenditures  Effeetive  Eirepower 

Wasted  Expenditures 

Wasted  Eirepower 

Engage  Decision  Delays, 

Hostile  Engage  Prob 

Track  Range,  Engage  Range,  Coordination  Efficiency 
Eratricide  Engage  Prob,  Duplicate/Ealse  Engage  Prob 

TIQ  Decision  Range  Traek  Range 

Duals  Merges  Swaps  ID  Errors  ID  confliets 

Connectivity  Data  Exchange  Data  Registration 

Info  Management  Track  Integration  Unit  Tactical 
Situation  Awareness  (TSA) 
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Consequences  of  Duals  and  ID  Errors 

Vary  Between  Engagement  Zones 


Overlapping 
Coverage  , 
Zones 


Duplicates,  Fratricide 
Unlikely  CAP 


/  Area 
[Defense  vs 
ASM 


Duplict 

MoreJi 


>elf  Defense . 


Area 

Defense  vs 
ASM 


>elf  Defense 


sSelf  Defense 


Area 

Defense  vs 
Fighter  ^ 


Duplicates  More  likely, 
Fratricide  Unlikely 

••^^rveillance  Zone 


Duplicates,  Fratricide 
Not  possible 


Coverage  (and  situation) 
determine  opportunity 
for  errors  to  result  in 
engagements 
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Topics 


•  Navy  AO  As 

•  Interoperability  Impact  on  System  & 
Mission  Capabilities 

•  Current  Challenges 


Purpose:  Describe  Interoperability  Implications  to  AOAs 


5/30/01 


13 


Challenges 


•  Testing  and  data  collection  is  a  major  challenge 

-  Testing  tailored  to  Interoperability  needed  -  otherwise  too  many  other  variables 

-  Highly  calibrated  testing  needed  -  all  objected  of  interest  must  be  geo-located 

-  HWIL  facilities,  like  DEP/JDEP,  will  help  significantly 

•  Decision-Making 

-  Decision  making  model  must  be  developed 

-  Decision-making  data  and  relationships  are  sparse,  at  best  -  must  collect 

-  Engagement  through  force  Models  do  not  include  decision  making 

•  Force  Models  do  not  include  interoperability 

-  Engineering  relationships  (Correlation,  sensor  performance,  ...relationships  to  dual 
tracks,  ID, ...) 

-  Analytical  relationships  (dual  tracks,  ID, ..  Impact  on  effectiveness) 

•  Fratricide  Models  and  Relationships  need  development 

•  Scenarios  must  have  appropriate  information  to  provide  inputs  to 
interoperability  estimating 


Interoperability  Analysis  and  its  impacts  are  just  beginning 


Summary 


•  A  Force  Interoperability  Methodology  has  been  established  (for  Air 
Defense)  for 

-  Defining  the  Metrics 

-  Relating  Metrics  to  Warfighting  Attributes 

-  Baselining  Force  Interoperability  Performance 

•  Force  Interoperability  measures  must  be  incorporated  into  the 
processes  for  decision-making, 

-  AOAs 

-  CRDs,ORDs 

-  TEMPS  and  APB 


Units  Needed  to  Win  Time  Needed  to  Win 

Red  Losses  Blue  Units  Lost  Fratricide 


PRA  Kills  Leakers 

Weapon  Efficiency  Fratricide  losses 


Layers  Employed  Engage  Pk 

Expenditures  Effective  Firepower 

Wasted  Expenditures  Wasted  Firepower 

Engage  Decision  Delays,  Hostile  Engage  Prob 

Track  Range,  Engage  Range,  Coordination  Efficiency 
Fratricide  Engage  Prob  ,  Duplicate/False  Engage  Prob 

TIQ  Decision  Range  Track  Range 

Duals  Merges  Swaps  ID  Errors  ID  conflicts 


Connectivity  Data  Exchange  Data  Registration 

Info  Management  Track  Integration  Unit  Tactical 
Situation  Awareness  (TSA) 
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Backup 


Analysis  of  Alternatives  Information  Flow 


Process 


Product 


OPNAV 

-  Mission  Need 

-  Preliminary 
Performance 
Requirements 

SYSCOM/Lab 
-Technical 
Characteristics/ 
Alternatives 
-Analytic  models 
-Cost  Estimates 


-Threat 


Contractors 

-Concepts 

-Technology 

-Historical 
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experience 


Analysis  of 
Alternatives 


-  Analysis 

-  Judgment 

-  Insight 


A 


Key  Performance) 
Parameters 


Other 

Performance 

Parameters 


Critical 

System  (logistic) 
Parameters 


•Measures  of 
Effectiveness 
•Measures  of 
Performance 

Cost 

Objectives 


Milestone 

Documents 


ORD/TEMP/APB 


ORD 


ORD/TEMP/APB 


TEMP/APB 


APB 
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Operational  Limitations 
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Interoperability  Terminology 


Battle  Force  Interoperability:  The  ability  of  two  or  more  units  to  share 
information  to  improve  the  effectiveness  of  combined  units  (the  force) 
over  units  operating  independently. 


Between  Force  Units  and  among  Forces 


JCS:  Interoperabiiity:  Ability  of  systems,  units,  or  forces  to  provide 
services  to  and  accept  services  from  other  systems,  units,  or  forces  and 
to  use  the  services  so  exchanged  to  enable  them  to  operate  effectively 
and  achieve  the  assigned  missions. 


5/30/01 


Between  Systems  within  a  Unit,  Force  Units, 
and  among  Forces 
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•  Measure  Battle  Group 
Interoperability 

-  Developmental  Testing 

-  Operational  Testing 

-  Root  Cause  Analysis 

•  73  Measures  of  Performance 
(MOP)  Created  Across  7  Measures 
of  Effectiveness  (MOE)  Levels 


DDG  57 


DD  978 
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ASN  Metric 


Metric 


Dual  Tracks 


ID  Differences 


Track 

Accuracy 


_ Sub  Metric _ 

Percent  TADIL  Dueled 
Percent  Local  Dueled 
TADIL  Tracks  Per  Object 
Tracks  Per  Object _ 

ID  Difference  Event  Rate 
Percent  ID  Difference 

Remote  XY  Accuracy  (ft) 
Remote  Altitude  Accuracy 

TO _ 

Local  XY  Accuracy  (ft) 

Local  Altitude  Accuracy  (ft) 


ID  ID  Correctness  Friend  % 
Correctness  ID  Correctness  Neutral  % 
ID  Correctness  Hostile  % 


mol 
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Sub  Metric  Mapping 


Metric 


Track  File 
Consistency 


Track 

Number 

Stability 


Sub  Metric 

Percentage  Time  Target  of 
Interest  on  Net 
Total  Consistency  % 

Link  Track  Number 
Consistency  % 

Position  Consistency  % 

ID  Consistency  % _ 

IFF  Consistency  % _ 

Engagement  Status 

Track  Number  Change  Rate 
per  Hour 


s  Are  Mo$^^portant  in 
Interoper^ 
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Force  Interoperability  Impacts  Hierarchy 


•  Single  Attack 

-  PRA 

-  Loss  Rates 

-  Kill  Rates 

-  Expenditure  Rates 

Engagement  Situation  (Multiple 
Attacks  and  Missions) 

Losses 

Expenditures 
Fratricide  Rates 


Campaign  (Multiple  Engagement 
Situations) 

Arrival  Rates 
Departure  rates 
Attrition 
Availability 
Endurance 
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Attack  Type  2 


Air  launched 
Subsonic  AS  CM 
vs  CVBG 


Day  5 

0600-1200Z 

12  Red  Attacks 
28  Blue  Attacks 


Scenario  5 
80  Days 
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Estimated  Translation  From 

Engagement  Situation  to  Campaign  Inputs 


Interoperability  Cases 
Good  RFT  _ 


ASCM  Defense  Lavers 
Self  Defense  Area  Defense 


Outer  Air  Battle 


Duals:]  0% 

Delay  Decision:  none 
Cap  Availability:  75% 

ID  errors:  20%  Hostile,  5%  Friend 


Shoot  Duals  (  Today  ”  Parametric  Goals 


BFI) 


Duals:  40%.  100%  engaged 
Delay  Decision:  none 
Cap  Availability:  50% 

ID  errors:  60%  Hostile,  20%  Friend 


Delay  Decision 


ASCIET  estimates 


Duals:  1 40%  I 

Delay  Decision:  1  layer  to 
resolve  duals 

Cap  Availability:  50% _ 

ID  errors:  60%  Hostile,  20%  Friend. 


12 


CAP  available: 
MCMs,  CVBG, 
^PREPO,  TBMD 


7^ 


CAP  available: 
CVBG,  TBMD 


'  'A  ^  r^. 


CAP  available: 
A:VBG,  TBMD 
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Repeatability/  Cases  /  New  Systems  Eval... 


Fidelity  /  Cost  /  Run  time 
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Analysis  of  Alternatives 


REQUIREMENTS 


Studies 

Analysis 

CINCs 

Threat  change 
Technology 


— - 

Mission  I 
Need  1 

Statement  1 
(MNS)  1 

— 

Materiel  Alternatives 

_ k 

ACQUISITION  (ASN  RD&A) 


Program  Decision  Meeting 
(PDM) 


Acguisition 

Decision 

Memorandum  (ADM) 


Measures  of 
Effectiveness 
and  Performance 
(MOE/MOP) 


- J-  *  IJ. 


Analysis  of  Alternatives  I  AO  A) 

Cost  Analysis  Improvement  Group 
(CAIG)  analyzes  performance/cost 
tradeoffs 

Makes  solution  recommendations 
Makes  Key  Performance  Parameter 
(KPP)  recommendations 


ave 


Subieguent  Phases/ 
Mil^tone  Decisions 


"Key  ^ 

Performance 

Parameters 


Milestone  B 
Approved 
Acguisition 
Program 
Baseline  (APB)J 
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AOA  INITIATION,  ANALYSIS,  AND  APPROVAL  PROCESS 


INITIATION 


PREPARATION: 

ACAT  I,  II,  III  -  SPONSOR 


PREPARATION: 

ACAT  IV-  PROGRAM 
_ MANAGER 


COORDINATION: 

ACAT  I  -  INTEGRATED  PRODUCT 
TEAM  (IPT) 

ACAT  11,111  -  IPT 


PROPOSAL  REVIEW: 

-  N810/DASN 


APPROVAL: 

N8  and  ASN(RD&A)  or 
designee:  (ACAT  I, II 
N8  and  MDA  (ACAT  III) 


IPT  RECOMMENDS: 

-  ANALYSIS  DIRECTOR  (BY  NAME) 

-  ANALYSIS  TEAM  (BY  ORGANIZATION) 

-  SCHEDULE 

-  COST  ESTIMATE 

-  ISSUES  TO  BE  ADDRESSED 


APPROVAL: 

ACAT  IV  -  MDA  and 
SPONSOR 


ANALYSIS 


ANALYSIS  TEAM  (nominal) : 

DIRECTOR: 

-  ASN{RD&A)  Rep 

ANALYSIS  PLAN  : 

{  FERDC/SYSCOM/LABS/CONTRACTOR) 

-  SPONSOR  Rep 

-  ISSUES 

-  SYSCOM/PEO/DRPM  Rep 

-  ALTERNATIVES 

-  PLAN/SUPERVISE  STUDY 

-  N81  REP  -  N80  Rep 

-SCENARIOS  (N812) 

-  N091  REP  -  N4  Rep 

-MODELS  {N812) 

-  COORDINATE  FUNDING  WITH 

-  N2  REP 

-  MOEs  (N812/N091) 

PROGRAM  MANAGER 

-  PM  Rep 

-  WORK  PLAN 

-  OTHERS  AS  APPROPRIATE 

- POA&M 

*  CNA  *  USMC  *DON  CIO  Rep 

*  WARFARE  CENTERS 

I  (NOTE:  ANALYSIS  TEAM  AT  DISCRETION  OF  MDA  FOR  ACAT  IV)  I 

APPROVAL 
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Test  Events  to  Date 


AFWTF  Constructive 
Geography 


Land-based  tests: 


CONTENT  BGIT  (JUN  00) 
ISET  (JUN  00) 

CEC  DEP  (JUL  00) 

BFIR  DEP  (DEC  00) 

JFK  BGIT  (JUN  01) 


ASCM 

Launcher 


Fighter/Bombers 


ASCM  Launche 


At  Sea  Testin 


CEC  Underway  8  (APR  00) 
CEC  Underway  9  (MAY  00) 
CEC  Underway  10  (SEP  00) 
CONBGSIT  (OCT  00) 

CEC  Underway  11  (DEC  00) 
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Example  “Engagement”  Actions  and  Layers 

Example  Fighter  Launched  Missile  Scenario 


INTERVENTIONS 
Counter  Surveillance  Assets 

Attack  Recon  Assets 
Cover  and  Deception 
Counter  C  3  I  Comms 
Kiii  Piatform 

Counter  Acquisition 
Counter  Targeting 
Counter  Launch 
Kiii  Weapon 

Counter  Weapon  Comms 
Distraction 


Seduction 
Counter  Fuze 
Counter  Weapon  Effects 
SMm  Capabiiity 


EXAMPLE  ACTION  LAYERS 

1  Deterrence  (Strike  Power) 

2  Fiexibiiity  and  Standoff  (Strike  range) 

3  Attack  Recon  Assets  (Strike,  CAP) 

4  Cover  and  Deception  (Decoys,  Tactics,  Sig  Controi) 

5  Counter  C3I  Comms  (EW,  Strike) 

6  Kiii  Launch  Piatform  (CAP) 

7  Kiii  Launch  Piatform  (Pickets  &  ER  SAM) 

8  Counter  Acquisition  (EW,  Sig  Controi) 

9  Counter  Targeting  (EW,  Sig  Controi) 

10  Counter  Launch  (EW,  Sig  Controi) 

11  Kiii  Weapon  (Pickets  &  ER  SAM) 

12  Kiii  Weapon  (Screen  &  Area  SAM 

13  Counter  Weapon  Comms  (EW) 

14  Distraction  (EW  ,  Sig  Controi) 

15  Kiii  Weapon  (  Seif  Defense  SAM 

16  Kiii  Weapon  (Last  Ditch  Weapon  (CIWS,  guns)) 

17  Seduction  (EW  ,  Sig  Controi) 

18  Counter  Fuze  (EW,  Sig  Controi) 

19  Counter  Weapon  Effects  (Armor,  Redun,  Sig  Controi) 

20  Retain  Capabiiity  (Redun,  Damage  Controi) 
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AO  A  Value  Added 


•  Basis  for  requirements 

•  Builds  the  basis  for  Interoperability  metrics  and  related  performance  of 
the  alternatives 

•  Constructs  meaningful  acquisition  alternatives,  adapting  existing  and 
proposed  systems 

•  Expresses  the  Program  boundaries 

•  Defines  acquisition  constraints 

•  Ties  ongoing  analysis  to  existing  testing;relationships  to  ORD  parameters 
and  performance  modeling 

•  First  Detailed  Cost  estimated  based  on  Selected  Option 
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Analysis  of  Alternatives 
Formal  Process 


•  Scope  of  Analysis 

-  Initiates  the  AOA  Process 

-  Must  be  approved  prior  to  initiation  of  analysis 

-  OSD(PA&E)  provides  detailed  input  for  ACAT  I 

•  Analysis 

-  Follows  process  developed  in  Scope  of  Analysis 

-  Reviews  via  Oversight  Boards  representing  key  acquisition,  requirements,  and 
fleet  offices 

•  Approval 

-  Required  briefings  dependent  on  ACAT  level  of  program 

-  Approved  by  Milestone  Decision  Authority  and  N8  (except  ACAT  IV  -  then  MDA 
&  Program  Sponsor) 
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Battle  Force  Interopability 

Status 


-In  Development  (TAMD  and  Power  Projection) 
»  Metrics  Heirarchy 
»  Baselining  Performance 
»  Relating  MOPs  to  Warfighting 

-  Testing:  Scenarios/  Data  Extraction/  Analysis 
tools 

-  Modeling  Interoperability  Affects 


Interoperability  Analysis  and  its  impacts  are  just  beginning 


BFI  Terminology 


VWT1T1]  nvi  ■  riwiiT 


TiSfaiicfa 


Battle  Force  Interoperability:  The  ability  of  two  or  more  units  to  share 
information  to  improve  the  effectiveness  of  combined  units  (battle  force) 
over  units  operating  independently. 


Decision  Time  Available  System  Time  Requirements 


Time 
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Testing  Considerations 


Test  Event  Factors  Exacerbate  System 
Interoperability 

•  Training 

•  Equipment  Reliability 

•  Operator  Action  variability 

•  Scenarios 

•  DT/OT^ 

•  Sensor  &  System  Limitations 

Land-Based  Testing  (DEP)  Provides 
Critical  System  Characterization 

•  OPTASK  LINK  Development  &  TADIL/CEC  Network 
Operations  Proficiency 

•  Benchmarking  Combat  System  Capabilities  and  Limitations 
for  At-Sea  Testing 
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^ ^ - 

BH^^^Engineering  (DT)  vs.  Training 
_ (OT) _ 

•  Merging  these  environments  introduces 

challenges  to  hoth  engineer  and  operator 

•  Development  tests  focused  on  engineering: 

•  Provided  data-friendly,  measurable,  and  repeatable  scenarios 

•  Although  these  events  tested  CEC  operational  functionality, 
crew  training  deficiencies  arose 

•  Attention  turned  to  operator  training 

•  Developing  crew  proficiency  in  this  new,  three  network 
environment  became  vital 

•  Varying  nature  of  operations  introduced  data  analysis 
complexities 


Training,  equipment  reliabiiity,  operator  actic 


Leveraging  Interoperability  Analyses 


•  Scenarios  need  to  be  conducted  within  combat  systems  and  sensor 
limitations 

•  The  DEP  provides  the  test  bench  for  baselining  combat  system 
limitations 


At-sea  testing  must  now  be  synchronized  to  benchmark  combat 
systems  limitations: 

•  Types  of  ships  and  aircraft 

•  Aegis/A  CDS  baselines 

•  C2P  models 

•  Hardware/software/firmware  loads 


•  AWS 

•  SGS 

•  CEC 

•  C2P 

•  Link  architecture 
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0V-6C  CVBG  D  Day  Operations-  TACSIT  3 
(SSC  Time  Critical  Target  Development-  SC  Attack  Thread) 

NTM  Theater  Tactical 

Time  Sensors  Sensors  JFACC  CVBG  Surface  Combatant  Sensor 


Recipient 
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ASN  (CHENG)  TCS  Functional  Breakdown 


NFN  Functions  noted  by: 


